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MICSS  Class  Experiments  and  Industry  Survey  Analysis1 


Parbati  Ray,  Advisors:  Thomas  Bellocci  and  Shimon  Y.  Nof 


ABSTRACT 


Two  surveys  have  been  developed  to  ascertain  the  information  assurance  requirements  of 
networked  enterprises.  The  surveys  give  an  insight  into  how  inter- networked  companies  use 
their  ERP  systems,  what  their  current  policies  maybe  with  respect  to  information 
management,  and  what  their  security  and  assurance  problems  maybe. 

The  surveys  focus  on  the  views  of  the  information  manager  of  the  firm  and  on  the  department 
managers  of  those  departments  that  depend  mostly  on  their  information  systems  for  smooth 
running. 

The  survey  questions  have  been  based  on  quantitative  analysis  done  by  experiments  using  an 
ERP  software  simulator,  MICSS  (Management  Interactive  Case  Study  Simulator).  The  logic 
and  procedures  used  to  develop  the  surveys  has  been  presented.  The  results  obtained  from  the 
analysis  of  the  survey  replies  will  enable  the  design  of  autonomous  agents  and  active 
protocols  to  help  companies  automatically  assure  their  information. 


1  This  work  was  supported  by  sponsors  of  the  Center  for  Education  and  Research  in  Information  Assurance, 
Purdue  University 


PART  A 


Design  of  Experiment: 

We  have  decided  to  study  4  factors  in  this  experiment. 

Factor  1: 

Dataset ;  with  4  levels:  Prices,  QLT  (Quoted  Lead  Time),  Batch  Size,  and  Order  Levels. 

Factor  2: 

Failure  type ;  with  2  levels:  “wrong  information”,  and  “delayed  information” 

Factor  3  (nested  in  “wrong  information”): 

Error  size ;  with  2  levels  “value  doubled”,  and  “value  halved”. 

Factor  4  (nested  in  “delayed  information”): 

Length  of  delay,  with  2  levels  “1  quarter”,  and  “2  quarters”. 

The  observations  haven’t  been  analyzed  like  a  nested  design.  We  didn’t  need  all  the 
information  given  by  a  nested  design  analysis.  For  simplicity  and  time  saving,  we  have 
used  single  ANOVAs  to  compare  each  time  two  different  scenarios. 

For  each  dataset,  the  following  comparisons  are  presented: 

Dataset  delayed  1  quarter  /  Baseline  policy  (for  profit). 

Dataset  delayed  2  quarters  /  Baseline  policy  (for  profit). 

Dataset  wrong  half  /  Baseline  policy  (for  profit). 

Dataset  wrong  double  /  Baseline  policy  (for  profit). 

The  datasets  are  presented  in  this  order:  Prices,  QLT,  Batch  Size,  Order  Level. 

Analysis  on  the  DDP  could  not  be  done  because  of  unavailable  data. 

Summary: 

Prices 

Fig.Al  -  Dataset  delayed  1  quarter  /  Baseline  policy  (for  profit). 

Dataset  delayed  2  quarters  /  Baseline  policy  (for  profit). 

Fig.A2  -  Dataset  wrong  half  /  Baseline  policy  (for  profit). 

Dataset  wrong  double  /  Baseline  policy  (for  profit). 

QLT 

Fig.A3  -  Dataset  delayed  1  quarter  /  Baseline  policy  (for  profit). 

Dataset  delayed  2  quarters  /  Baseline  policy  (for  profit). 

Fig.A4  -  Dataset  wrong  half  /  Baseline  policy  (for  profit). 

Dataset  wrong  double  /  Baseline  policy  (for  profit). 

Batch  Size 

Fig.A5  -  Dataset  delayed  1  quarter  /  Baseline  policy  (for  profit). 

Dataset  delayed  2  quarters  /  Baseline  policy  (for  profit). 

Fig.A6  -  Dataset  wrong  half  /  Baseline  policy  (for  profit). 

Dataset  wrong  double  /  Baseline  policy  (for  profit). 

Order  Level 

Fig.A7  -  Dataset  delayed  1  quarter  /  Baseline  policy  (for  profit). 

Dataset  delayed  2  quarters  /  Baseline  policy  (for  profit). 

Fig.A8  -  Dataset  wrong  half  /  Baseline  policy  (for  profit). 

Dataset  wrong  double  /  Baseline  policy  (for  profit). 


Notations: 

“D”  means:  Hie  two  scenarios  give  significantly  different  results. 

“D  means  that  the  performance  with  information  failure,  for  profit  or  DDP,  is  worse 
than  with  the  baseline  policy. 

“D  +“  means  that  the  performance  with  information  failure,  for  profit  or  DDP,  is  better 
than  with  the  baseline  policy. 

“S”  means:  The  two  scenarios  give  significantly  similar  results. 


Fig.Al  -  Prices; 

Dataset  delayed  1  quarter  /  Baseline  policy  (for  profit) 
Dataset  delayed  2  quarters  /Baseline  policy  ( for  profit ) 
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Fig.  A1  shows  that  a  delay  in  Prices  affects  the  Profit  in  a  significant  manner.  The  data  for  quarter 
2  and  3  for  a  two-quarter  delay  is  missing  and  hence  not  plotted.  Table  A1  summarizes  the  effects 
of  delayed  information  on  prices. 


Fig.A2  -  Prices; 

Dataset  Wrong  half  /  Baseline  policy  (for  profit ) 
Dataset  Wrong  double  /  Baseline  policy  ( for  profit) 
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As  seen  from  Fig.  A2,  the  amount  by  which  the  prices  are  changed  is  insignificant,  the  effect 
is  essentially  the  same.  The  range  analysis  shows  that  the  ranges  are  the  same. 


Fig  A3  -  QLT; 

Dataset  delayed  1  quarter  /  Baseline  policy  (for  profit) 
Dataset  delayed  2  quarters  /  Baseline  policy  ( for  profit ) 


QLT  Delayed 


Correct 
*  2  qtr  delay 
1  qtr  delay 


Table  A3 


QTR  1 

QTR  2 

QTR  3 

QTR  4 

2  qtr  delay  vs.  correct 

D- 

D- 

D- 

D- 

1  qtr  delay  vs.  correct 

D- 

D- 

D- 

D- 

Fig.  A3  shows  that  a  delay  in  QLT  affects  the  Profit  in  a  significant  manner.  There  is  a  significant 
decrease  in  the  Profit  if  the  information  is  delayed  a  quarter/two  quarters.  The  effects  are 
summarized  in  Table  A3. 


Fig.A4  -  QLT; 

Dataset  Wrong  half  /  Baseline  policy  (for  profit ) 
Dataset  Wrong  double  /  Baseline  policy  ( for  profit ) 
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Table  A4 
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It  can  be  seen  from  Fig.  A4  that  a  larger  error  in  value  can  cause  a  major  decrease  in  profit, 
however  a  small  change  will  affect  it  only  in  the  long  run.  A  range  analysis  shows  that  the 
half  and  correct  values  are  actually  almost  the  same  for  the  first  2  quarters.  Ranges  are  shown 
only  for  those  quarters  where  the  averages  are  very  close.  This  is  done  to  see  if  the  ranges  of 
the  information  overlap  or  not. 


Fig.A5  -  Batch  Size; 

Dataset  delayed  1  quarter  /  Baseline  policy  (for  profit) 
Dataset  delayed  2  quarters  /  Baseline  policy  ( for  profit) 
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Fig.  A5  shows  that  Batch  Size  is  mostly  unaffected  by  a  delay.  A  sensitivity  Analysis  makes 
the  point  clear.  It  can  be  seen  from  the  ranges,  that  the  values  are  almost  similar  after  period 
one  for  a  2- quarters  delay,  but  are  different  for  a  1- quarter  delay.  This  implies  that  the  average 
values  of  the  information  due  to  1- quarter  delay  and  2- quarters  delay  are  the  averages  of  the 
same  range  of  values,  indicating  that  there  is  no  affect  of  time  delay. 


Fig.A6  -  Batch  Size; 

Dataset  Wrong  half  /  Baseline  policy  ( for  profit) 
Dataset  Wrong  double  /  Baseline  policy  ( for  profit ) 
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It  can  be  noted  that  a  large  increase  in  Batch  Size  has  no  affect  on  profit,  but  a  small  decrease 
will  change  profits  drastically.  A  range  analysis  shows  that  the  correct  and  double  values 
coincide. 


Fig.A7  -  Order  Level; 

Dataset  delayed  1  quarter  /  Baseline  policy  (for  profit) 
Dataset  delayed  2  quarters  /  Baseline  policy  ( for  profit) 
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A  2  quarters  delay  on  Order  Level  has  an  impact  on  the  functioning  of  the  company  and 
lowers  the  Profit  in  the  long  run. 

In  average,  a  1  quarter  delay  on  Order  Level  doesn’t  affect  the  Profit  of  the  company. 
However,  it  can  be  seen  from  the  ranges  analysis  that  the  variability  in  the  Profit  of  the 
company  is  much  higher  with  1  quarter  delay  scenarios  than  with  correct  information.  So, 
effects  of  a  1  quarter  delay  are  quite  unpredictable,  and  thus  they  are  dangerous. 


Fig.A8  -  Order  Level; 

Dataset  Wrong  half  /  Baseline  policy  (for  profit) 
Dataset  Wrong  double  /  Baseline  policy  (for  profit) 
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When  the  order  level  is  doubled,  there  is  a  decrease  in  profit  but  halving  the  value  of  correct 
information  has  no  significant  effect,  this  may  be  seen  from  the  range  analysis  also.  The  range 
analysis  shows  that  the  ranges  of  the  average  values  are  the  same,  hence  the  averages  truly 
overlap. 


PART  B 


School  of  Industrial  Engineering 
Purdue  University 
PRISM  Lab 
CERIAS  Project 


To  whom  it  may  concern: 

We  are  writing  to  you  with  regards  to  a  project  undertaken  by  our  research  team  and  funded 
by  CERIAS.  The  objective  of  our  project  is  to  improve  information  assurance  with  a  focus  on 
enterprise  information  systems. 

In  today’s  inter- networked  companies,  the  integration  of  the  business  operations  has 
created  new  needs  regarding  the  administration  of  distributed  information  systems,  hrformation 
security  is  no  longer  sufficient.  The  information  exchanged  must  be  both  secure  and  significant.  At  the 
same  time,  employees  have  difficulties  to  operate  with  these  increasingly  complex  information 
systems,  and  look  upon  security  practices  as  slowing  them  down  in  performing  their  jobs. 

The  aim  of  our  research  project  is  to  address  the  above  concerns  by  designing  autonomous 
agents  and  active  protocols  to  help  companies  in  automatically  assuring  their  information.  We  define 
information  assurance  as  the  combination  of: 

•  Information  security,  which  means  protecting  information  from  malicious  threats  and 
damage  due  to  external  or  internal  sources. 

•  Information  integrity,  which  should  be  understood  as  permanency  of  the  information 
during  communications  and  storage. 

•  Information  significance,  which  refers  to  the  value  that  the  intended  user  can  get  out  of 
the  information  when  s/he  receives  it. 


To  help  this  effort,  it  is  necessary  to  first  identify  information  assurance  requirements  for  ERP 
and  related  information  systems,  the  parameters  that  could  significantly  affect  information  assurance, 
and  the  potential  threats  of  assurance  failure  on  the  performance  and  profits  of  a  company. 

Surveys  have  been  developed  for  the  departmental  and  the  information  system  managers’ 
viewpoints,  to  understand  how  inter-networked  companies  use  their  ERP  and  related  information 
systems,  what  their  current  policies  are  regarding  information  management,  and  what  their  security 
and  assurance  problems  may  be.  We  believe  that  your  company  can  benefit  from  looking  into  these 
issues. 

We  would  highly  appreciate  it,  if  you  could  help  us  perform  this  study  more  efficiently  by 
filling  in  the  attached  survey  questionnaires.  We  have  included  two  surveys,  one  to  be  filled  by  the 
information  systems  manager  and  the  other  to  be  filled  by  the  various  department  managers  of  your 
company.  We  would  be  very  grateful  if  you  could  forward  them,  copies  of  the  same.  Example  answers 
have  been  included  in  order  to  clarify  questions  and  terminology,  but  please  feel  free  to  contact  us  via 
email  if  you  have  any  questions.  You  may  send  the  responses  to  the  surveys  either  by  email  or  by  post, 
by  the  end  of  April.  When  the  survey  analysis  is  complete,  we  will  gladly  share  the  results  with  you. 


Thank  you  very  much  for  your  cooperation, 

Thomas  Bellocci  and  Parbati  Ray  -  Research  Students 


Email  addresses:  bellocci@purdue.edu,  paro @ purdue.edu 

US  postal  address:  School  of  Industrial  Engineering 
Purdue  University 
Pr.  Shimon  Y.  Nof 
West  Lafayette,  IN  47907 


INFORMATION  SYSTEM  MANAGER  SURVEY  (filled  sample) 


Please  review  the  following  questions  and  answer  briefly.  Our  project  team  is  trying  to  establish  what 
information  assurance  is  actually  required. 

Assuring  information  means  having  a  secure  system  and  procedures,  which  guarantee  that 
information  is  secure,  and  the  information  keeps  its  integrity  and  significance  during  its  lifetime. 

Company:  Name  Respondent  Name :  _ 

Position:  c.,g„  Information  Analyst 


1)  Are  the  executive  board  and  senior  management  aware  of  the  importance  of 
information  assurance  for  the  smooth  functioning  of  the  company? 

□  Yes 

□  No 

2)  If  yes,  does  your  company  have  specific  policies  and  procedures  to  assure  your 
information? 


□  Yes 

□  No 

3)  Which  of  the  following  preventive  measures  does  your  company  employ  to  protect 
itself  from  external  threats? 


□  Anti  vims 

□  Hacking  watch 

□  Firewalls 

□  Encryption 

□  System  authorizations  (passwords,  access  restrictions) 

□  Other 

4)  Does  your  company  distinguish  between  different  datasets  in  its  information  system? 

□  Yes 

□  No 

5)  If  yes,  please  fill  in  Table  1  to  describe  some  of  the  typical  datasets  and  their 
characteristics. 

[See  Table  1] 


6)  Does  your  company  distinguish  between  different  groups  of  users  allowed  to  log  into 
its  information  system? 


□  Yes 

□  No 

7)  If  yes,  please  fill  in  Table  2  to  describe  the  different  groups  and  their  characteristics. 

[See  Table  2] 


8)  Does  your  company  have  a  general  data  maintenance  policy? 

□  Yes 

□  No 

9)  How  often  do  you  have  a  maintenance  session  for  the  data? 

e.g..  Once  every  quarter 


10)  What  data  do  you  monitor  regularly? 

e.g..  The  number  of  times  an  employee  logs  on  to  a  certain  ‘locked’  piece  of  information 


11)  How  can  your  company  handle  the  following  problems?  (Several  answers  are  possible) 


Category 

You  Can  detect 

You  Can 
prevent 

You  Can  recover 
from 

Transmission  failures 

Data  decay  during  storage 

Accidental  loss  of  data 

Low  quality  of  communication 
links 

System  crash 

Loss  of  data  due  to  system  crash 

Short  term  delays  (one  day) 

Long  term  delays 

Very  long  term  delays 

12)  What  do  you  estimate  is  the  damage  from  non-assured  information  on  your 
departments’  performance?  Please  circle  the  appropriate  answer. 


Relatively  insignificant  Somewhat  significant  Very  significant  None 


TABLE  1  (for  question  5) 
Please  fill  for  5-6  typical  datasets 


TABLE  2  (for  question  7) 

Please  fill  in  for  5-6  typical  datasets 


Group  name _ Users  in  the  group _ Assurance  features 

e.g.,  e.g.,  e.g., 

Project  Alpha  a)  All  the  employees  working  a)  Can  read  all  the  information 

on  project  alpha  concerning  project  alpha 

b)  Hie  clients  of  project  alpha  b)  Only  the  manager  of  the  project 


can  modify  the  data  of  project  alpha 
c)  Modification  are  done  once  a 
week 


Thank  you  very  much  for  participating  in  our  survey 


DEPARTMENT  MANAGER  SURVEY  (filled  sample) 


Please  review  the  following  questions  and  answer  briefly.  Our  project  team  is  trying  to  establish  what 
information  assurance  is  actually  required. 


Company:  Name  Respondent  Name(s): 

Department:  e.g..  Production  Planning 
Position:  e.g.,  Department  Manager 


1)  How  often  does  your  department  change  its  strategies  in  using  the  company’s  ERP 
and  related  information  systems?  (e.g.,  Strategies  may  include  planning  and  operational 
procedures,  decision  policies,  decision  logic  etc) 

□  At  regular  intervals  (Please  specify  the  length  of  the  interval):  e.g.,  every  month 

□  If  needed  (Please  describe  what  the  circumstances  are):  e.g.,  change  of  managing 
committee 


2)  At  the  time  of  changing  your  strategies,  have  you  ever  had  to  make  your  decisions  with 
missing  or  wrong  information?  Please  circle  the  appropriate  answer. 

□  Yes  :  often  occasionally 

□  No 

3)  If  yes,  which  information  was  missing  or  wrong?  [You  may  refer  to  your  most  recent 
project.] 

e.g..  The  size  of  production  batch  was  conveyed  incorrectly 

4)  Which  division  of  the  company  should  have  given  you  this  information? 

e.g..  Production 

5)  Could  you  have  waited  longer  to  get  this  information  before  making  changes  in  your 
strategies?  If  yes,  how  long?  [You  may  refer  to  your  most  recent  project.]  Please  circle 
the  appropriate  answer. 

□  Yes  :  a  few  hours  a  few  days  a  week  or  more  other  (please  mention) 

□  No 

6)  What  do  you  estimate  is  the  damage  from  non-assured  information  on  your 
departments’  performance?  Please  circle  the  appropriate  answer. 

Relatively  insignificant  Somewhat  significant  Very  significant  None 


7)  How  often  do  you  examine  the  relative  significance  (value)  of  the  information  you 
gather/  store/maintain?  Please  circle  the  appropriate  answer. 

Frequently  Occasionally  Only  at  system  change  Not  for  a  long  time 


8)  Why  were  you  not  completely  informed  at  the  time  of  making  decisions  to  change 
your  strategies?  [You  may  fill  in  more  than  one  column,  referring  to  your  cumulative 
experience.] 


t 

□  Sometimes  the 
information  was  not 
available. 

V 

□  Sometimes  the 
information  was 
delayed. 

* 

□  Sometimes  the  information 
was  disregarded. 

V 

V 

e.g., 

□  Nobody  measures  that 
information. 

□  You  are  not  allowed  to 
get  this  information. 

e.g., 

□  The  people  were 
late. 

□  Communication 
problems  occurred. 

e.g., 

□  The  source  was  not  reliable. 

□  The  value  was  unreasonable. 

□  The  information  was  not 
complete. 

□  The  format  of  the  information 
was  inappropriate. 

□  The  information  was  not 
accurate  enough. 

□  The  information  was  not 
relevant. 

9)  Which  type  of  data  is  most  often  affected  by  information  failures  like  delayed  or 
wrong  information? 


e.g..  Work  in  Progress.  Level  of  stocks 


10)  For  which  type  of  data  are  the  consequences  of  information  failure  the  most 
dangerous? 


e.g.,  Idle  Labor  and  Resources,  Selling  prices 

11)  Which  performance  measure  of  your  company  is  most  affected  when  an  information 
failure  occurs? 

e.g..  Customer  relations.  Profits 


12)  Other  comments  about  recurrent  problems  you  encounter  while  using  the  ERP  and 
related  information  systems  of  the  company: 

e.g..  This  has  become  a  vicious  cycle  and  our  department  is  forced  to  take  decisions 
without  correct  information 


Thank  you  very  much  for  participating  in  our  survey! 


Information  System  Manager  Survey  Analysis 

a:  Software  Company  (  Korea ) 
b:  Mechanical  Construction  (France) 
c:  Software  Company  (U.S) 
d:  Government  laboratory  (U.S) 
e:  Software  Company  (U.S) 
f:  Software  Company  ( Globed ) 
g:  Software  Company  (U.K) 
h:  Manufacturing  Company  (Hungary) 

I:  Bank  (France) 

J:  IT  Company  (U.S) 

Ql.  Are  the  executive  board  and  senior  management  aware  of  the  importance  of 
information  assurance  for  the  smooth  functioning  of  the  company? 

a)  Yes 

b)  Yes,  recent  development 

c)  Yes 

d)  Yes 

e)  Yes 

f)  Yes 

g)  Yes 

h)  No 

i)  Not  all 

j)  Yes 

Conclusion:  The  concept  of  information  assurance  is  popular. 

Q2.  If  yes,  does  your  company  have  specific  policies  and  procedures  to  assure  your 
information? 

a)  Yes 

b)  Yes,  not  much  though 

c)  Yes 

d)  Yes 

e)  Yes 

f)  Yes 

g)  Yes 

h)  No 

i)  No 

j)  Yes 


Conclusion:  Yes 


Q3.  Which  of  the  following  preventive  measures  does  your  company  employ  to  protect 
itself  from  external  threats? 

a)  Antivirus  and  System  Authorization 

b)  AH 

c)  AH 

d)  AH 

e)  AH 

f)  Antivirus,  Firewalls,  Encryption,  System  Authorization. 

g)  Antivirus,  Firewalls,  Encryption,  System  Authorization 

h)  Antivirus,  System  Authorizations 

i)  Antivirus,  Firewalls,  System  authorizations,  Filter  of  attached  file  in  mails 

j)  Antivirus,  Firewalls,  System  autorizations 

Conclusion:  Most  of  the  companies  have  all  the  preventive  measures  listed.  Antivirus  and 
System  Authorizations  are  the  most  popular  ones. 

Q4.  Does  your  company  distinguish  between  different  datasets  in  its  information 
system? 

a)  Yes 

b)  Yes,  but  each  dataset  is  managed  differently. 

c)  Yes 

d)  Yes 

e)  No 

f)  Yes 

g)  Yes 

h)  Yes 

i)  Yes 

j)  Yes 

Conclusion:  Yes 

Q5.  Some  typical  datasets  and  their  characteristics. 

a)  Has  well-defined  assurance  features.  Data  from  a  certain  dataset  can  be  read  and 
modified  by  employees  of  that  department.  The  data  from  the  dataset  is  also  updated 
frequently,  ranging  from  once  a  day  to  all  the  time. 

b)  NOT  ANSWERED 

c)  The  data  from  datasets  can  be  read  and  modified  by  members  of  the  team 

d)  Guidelines  are  set  by  project  and  network  managers 

e)  NOT  ANSWERED 

f)  Has  a  standard  setup  of  audit  measures  for  separation  of  duties  and  also  have  their 
security  aligned  with  this  effort.  They  have  specific  tools  to  help  audit  each  dataset. 

g)  Daily  Backup,  Developer/Purchaser/Team/Team  leader  only  access  and  no  detection 
rights  depending  upon  the  type  of  dataset.  On  completion  data  is  stored  in  source 
control 

h)  Can  only  be  read  by  the  managers 

i)  Mostly  passwords  are  the  assurance  features 

j)  Only  certain  employees  and  transaction  can  access  and  change  data 


Conclusion:  Most  of  the  companies  allow  project  members  to  access  all  the  data  for  their 
respective  projects  and  audits  are  conducted  on  a  timely  basis. 

Q6.  Does  your  company  distinguish  between  different  groups  of  users  allowed  to  log  into 
its  information  system? 

a)  Yes 

b)  Yes 

c)  Yes 

d)  Yes 

e)  Yes 

f)  Yes 

g)  Yes 

h)  Yes 

i)  Yes 

j)  Yes 

Conclusion:  Yes 

Q7.  The  different  users  groups  and  their  characteristics. 

a)  Groups  are  on  the  basis  of  project  teams  and  team  members  can  read  all  the 
information  regarding  their  specific  project 

b)  NOT  ANSWERED^ 

c)  NOT  ANSWERED 

d)  Groups  are  project  based  and  the  users  are  those  people  who  have  a  ‘need  to  know’ 
about  the  project.  Project  and  network  managers  set  controls. 

e)  The  members  of  a  project  team  have  access  to  all  the  information.  Typically  they  have 
several  levels  of  access  including 

•  Read  Only 

•  Create/Change  but  no  delete 

•  Full  Access 

f)  Has  a  standard  setup  of  audit  measures  for  separation  of  duties  and  also  have  their 
security  aligned  with  this  effort.  They  have  specific  tools  to  help  audit  each  dataset 

g)  Daily  backup  and  access  is  given  to  only  those  involved  with  the  data/team 

h)  Only  managers  can  modify  data  of  the  production  plan.  Modification  takes  place  once 
a  week.  Modification  requires  the  CEO’s  permission. 

i)  It  depends  on  the  type  of  data.  But  most  of  the  assurance  features  are  passwords 

j)  Authorizations,  training  and  reporting  to  review  and  summarize  the  processes 
performed 


Conclusion:  All  project  members  can  read  the  data,  and  depending  on  the  project  managers 
and  network  controls,  the  have  added  access  and  power.  Back  ups  are  also  a  popular  practice. 


Q8.  Does  your  company  have  a  general  data  maintenance  policy? 


a)  Yes 

b)  NOT  ANSWERED 

c)  No 

d)  No 

e)  Yes 

f)  Yes 

g)  No 

h)  Yes 

i)  Don’t  know 

j)  No 

Conclusion:  Some  companies  do  and  some  don’t.  In  this  case  it  seems  that  the  companies 
have  a  general  data  maintenance  policy. 

Q9.  How  often  do  you  have  a  maintenance  session  for  the  data? 

a)  Whenever  necessary 

b)  Whenever  necessary,  usually  every  6  months 

c)  Whenever  necessary 

d)  Every  project  has  it’s  own  policy,  some  more  tight  and  others  lose  because  of  the  non 
criticality  of  the  data 

e)  Irregular 

f)  The  Data  Maintenance  Group  does  maintenance  and  they  maintain  customer,  material 
and  vendor  master  data  as  required.  Various  departments  perform  other  maintenance 
functions  and  they  each  have  specific  tasks  and  responsibilities. 

g)  NOT  ANSWERED 

h)  Once  every  half  year 

i)  NOT  ANSWERED 

j)  Validate  backups  quarterly,  continuous  updating  of  master  data 
Conclusion:  Whenever  necessary. 

Q10.  What  data  do  you  monitor  regularly? 

a)  Disk  usage  status  for  users 

b)  Completion  of  batch,  availability  of  servers,  number  of  packages  on  some  packages  or 
databases,  number  of  calls  to  the  hot- line 

c)  Security  related  information 

d)  Depends  on  the  sensitivity  of  the  project;  some  areas  log  all  data  while  others  don’t 

e)  At  present  very  little,  but  the  policy  is  being  changed  and  a  more  active  approach  will 
be  taken  in  the  near  future 

f)  Right  now  monitoring  is  simple,  it  only  tells  them  who  logged  in  and  failed  to  do  so.  If 
a  data  is  updated,  it  cannot  be  recorded.  But  the  goal  is  to  try  and  add  additional 
monitoring  devices  that  assist  Internal  Audit  with  controls  and  manual  audit 
procedures. 

g)  NOT  ANSWERED 

h)  Data  integrity  of  supply  chain  modules 


i)  NOT  ANSWERED 

j)  Typically  there  are  authorizations  made  to  protect  the  data  sets 


Conclusion:  Depends  on  the  company,  but  mostly  depends  on  the  sensitivity  of  the  project. 
Some  companies  do  not  have  such  audit  procedures  and  hope  to  implement  one  in  the  near 
future. 


Qll.  How  can  your  company  handle  the  following  problems? 


Category 

You  Can  detect 

You  Can  prevent 

You  Can  recover 
from 

Transmission  failures 

a)  Yes 

a)  - 

a)  Yes 

b)  Partially 

b)  No 

b)  Do  it  again 

c)  Yes 

c)  - 

c)  Yes 

d)  Yes 

d)  - 

d)  Yes 

e)  Yes 

e)  Yes 

e)  Yes 

f)  Yes 

f)  Yes 

f)  Yes 

g)  - 

g)  - 

g)  - 

h)  - 

h)  - 

h)  - 

i)  Yes 

i)  - 

i)  Yes 

j)  Yes 

j)  - 

j)  - 

Data  decay  during  storage 

a)  Yes 

a)  - 

a)  Yes 

b)  No 

b)  - 

b)  Try  previous 

c)  Yes 

c)  - 

storage 

d)  Yes 

d)  - 

c)  Yes 

e)  - 

e)  - 

d)  Yes 

f)  No 

f)  Yes  (by 

e)  Yes 

g)  - 

tape 

f)  Yes,  but  some 

h)  - 

rotation) 

data  may  be 

i)  yes 

g)  - 

lost 

j)  - 

h)  - 

g)  - 

i)  - 

h)  - 

j)  - 

i)  Yes 

.i)  - 

Accidental  loss  of  data 

a)  Yes 

a)  - 

a)  Yes 

b)  By  the 

b)  - 

b)  Try  previous 

consequ 

c)  - 

storage  and 

ences 

d)  Yes 

log  files 

c)  Yes 

e)  Yes 

c)  Yes 

d)  Yes 

f)  No, 

d)  Yes 

e)  - 

except  to 

e)  Yes 

f)  No 

recover 

f)  No 

g)  - 

from  prior 

g)  Backup 

h)  - 

back  up 

h)  - 

i)  Yes 

g)  - 

i)  Backup 

j)  - 

h)  - 

j)  - 

i)  - 

i)  - 

Low  quality  of  communication 
links 

a)  Yes 

b)  Yes 

c)  Yes 

d)  Yes 

e)  Yes 

f)  Yes 

g)  - 

h)  - 

i)  Yes 

j)  Yes 

a)  - 

b)  Preventive 
Survey 

c)  - 

d)  - 

e)  Yes 

f)  Require  a 
particular 
speed 

g)  - 

h)  - 

i)  Yes 

j)  - 

a)  Yes 

b)  N/A 

c)  Yes 

d)  Yes 

e)  Yes 

f)  No 

g)  - 

h)  - 

i)  Yes 

j)  ' 

System  crash 

a)  Yes 

a)  - 

a)  Yes 

b)  Partially 

b)  Partial 

b)  Automatically 

c)  Yes 

redundanc 

(with 

d)  Yes 

y  of 

clusters)  or  by 

e)  Yes 

servers 

manual  repair 

f)  Yes 

(clusters) 

(more  general 

g)  - 

c)  - 

case) 

h)  - 

d)  Yes 

c)  Yes 

i)  Yes 

e)  Yes 

d)  Yes 

j)  Yes 

f)  Somewhat 

e)  Yes 

with 

f)  Yes  ,  back  up 

regular 

systems 

PM 

g)  Backup  -Hot 

g)  - 

standby 

h)  - 

h)  - 

i)  - 

i)  Yes 

j)  No 

j)  Yes 

Loss  of  data  due  to  system 
crash 

a)  Yes 

b)  By  the 
consequ 

ences 

c)  Yes 

d)  Yes 

e)  - 

f)  Yes 

g)  - 

h)  - 

i)  - 

j)  - 

a)  - 

b)  Partial  use 
of 

database 

transactio 

ns 

c)  - 

d)  - 

e)  Yes 

f)  Regular 
backups 

g)  - 

h)  - 

i)  - 

j)  Yes 

a)  Yes 

b)  Automatically 
for  a  part,  and 
with  saved 
storage  for 
the  other  part. 
With  human 
watching  on 
the  re-start 

c)  Yes 

d)  Yes 

e)  Yes 

f)  Yes,  but  may 
have  some 
lost  based  on 
incremental 
backup  or 
mirrored 
device  sync 
issues 

g)  - 

h)  - 

i)  Recover  with 
back  up 

j)  - 

Short  term  delays  (one  day) 

a)  Yes 

a)  - 

a)  Yes 

b)  - 

b)  - 

b)  - 

c)  Yes 

c)  - 

c)  Yes 

d)  Yes 

d)  - 

d)  Yes 

e)  Yes 

e)  Yes 

e)  Yes 

f)  No 

f)  No 

f)  No 

g)  - 

g)  - 

g)  - 

h)  - 

h)  - 

h)  - 

i)  Yes 

i)  - 

i)  yes 

j)  yes 

j)  yes 

j)  yes 

Long  term  delays 

a)  Yes 

a)  - 

a)  Yes 

b)  - 

b)  - 

b)  - 

c)  Yes 

c)  - 

c)  Yes 

d)  Yes 

d)  - 

d)  Yes 

e)  Yes 

e)  Yes 

e)  Yes 

f)  No 

f)  No 

f)  No 

g)  - 

g)  - 

g)  - 

h)  - 

h)  - 

h)  - 

i)  yes 

i)  - 

i)  yes 

j)  yes 

j)  yes 

j)  yes 

Very  long  term  delays 

a)  Yes 

a)  - 

a)  Yes 

b)  - 

b)  - 

b)  - 

c)  Yes 

c)  - 

c)  Yes 

d)  Yes 

d)  - 

d)  Yes 

Conclusion: 


e)  Yes 

e)  Yes 

e)  Yes 

f)  No 

f)  No 

f)  No 

g)  - 

g)  - 

g)  - 

h)  - 

h)  - 

h)  - 

i)  yes 

i)  - 

i)  yes 

j)  yes 

j)  yes 

j)  yes 

Category 

You  Can  detect 

You  Can 
prevent 

You  Can  recover 
from 

Transmission  failures 

Yes 

No 

Yes 

Data  decay  during  storage 

Yes 

No 

Yes 

Accidental  loss  of  data 

Yes 

No 

Yes 

Low  quality  of  communication 
links 

Yes 

Depends 

Yes 

System  crash 

Yes 

To  a  certain 

extent 

Yes 

Loss  of  data  due  to  system  crash 

Yes 

To  a  certain 

extent 

Yes 

Short  term  delays  (one  day) 

Yes 

No 

Yes 

Long  term  delays 

Yes 

No 

Yes 

Very  long  term  delays 

Yes 

No 

Yes 

Q12.  What  do  you  estimate  is  the  damage  from  non-assured  information  on  your 
departments’  performance? 

a)  Somewhat  Significant 

b)  Very  Significant 

c)  Somewhat  Significant 

d)  Relatively  insignificant 

e)  Somewhat  Significant 

f)  Relatively  insignificant 

g)  Relatively  Significant 

h)  Relatively  insignificant 

i)  Significant 

j)  Relatively  insignificant 


Conclusion:  Somewhat  significant,  however  some  companies  estimate  the  damage  from 
being  very  significant  to  relatively  insignificant. 


Department  Manager  Survey  Analysis 

a:  Software  Company  (Korea) 

b:  Mechanical  Construction  (France) 

c:  Software  Company  (France) 

d:  Government  Laboratory  (U.S) 

e:  Software  Company,  Software  Development(U.S) 

f:  Manufacturing  Company  (Israel) 

g:  Electronic  Company,  Midwest  (U.S) 

h:  Bank  (France) 

I:  IT  company  (U.S) 

Ql.  How  often  does  your  department  change  its  strategies  in  using  the  company’s  ERP 
and  related  information  systems? 

a)  At  regular  intervals;  Organizational  changes. 

b)  2  to  3  times  a  year;  for  new  projects,  approximately  5  a  year 

c)  Whenever  needed;  operational  issue,  change  in  business  plan,  business  event,  or 
request  by  the  management  team 

d)  Whenever  needed;  as  when  new  technology  (systems)  enters  the  market 

e)  Whenever  needed;  whenever  development  targets  change 

f)  Every  quarter;  on  the  division  steering  committee 

g)  If  needed.  If  the  processes  don’t  work,  it  is  fixed.  A  continuous  effort  is  made  in  trying 
to  improve  the  system,  but  little  is  achieved. 

h)  Reforcasting  every  6  months.  Changes  are  not  always  very  significant.  Otherwise 
when  need  be  like  in  the  case  of  important  deviations  in  actual  results  vs.  budget. 

i)  On  a  smaller  scale-  weekly,  on  a  larger  scale-  monthly 

Conclusion:  4  out  of  the  6  companies  say  that  they  change  their  policies  as  when  required. 

The  requirements  may  be  due  to  new  or  impending  projects/technology  and  development 
targets.  They  may  also  be  due  to  changes  made  by  the  management  team  w.r.t  the  operating 
methods. 

Q2.  At  the  time  of  changing  your  strategies,  have  you  ever  had  to  make  your  decisions 
with  missing  or  wrong  information? 

a)  Yes,  occasionally 

b)  Yes,  often 

c)  No 

d)  Yes,  often 

e)  Yes,  occasionally 

f)  Yes,  often 

g)  Yes,  occasionally 

h)  Yes,  Often 

i)  Yes,  occasionally 


Conclusion:  Yes,  often 


Q3.  If  yes,  which  information  was  missing  or  wrong? 


a)  The  purpose  of  the  hardware  purchasing  was  not  communicated  correctly 

b)  Missing  some  costs  (wrong  account  number) 

Differences  between  information  from  accounting  and  operations  about  number  of 
hours  of  work 

c)  N/A 

d)  The  capabilities  of  a  switching  device 

e)  Technologies/  timescales  for  development 

f)  The  market  fluctuations  and  the  integrative  effect  of  the  decisions 

g)  Even  if  we  get  data,  which  is  quite  difficult  to,  we  doubt  the  integrity  of  the  data 

h)  Unexact  margin;  analysis  of  rentability  by  customer,  sales,  product  could  have  been 
unexact. 

i)  Pricing  information 

Conclusion:  Various  answers,  depends  on  the  company  and  department  taking  the  survey. 


Q4.  Which  division  of  the  company  should  have  given  you  this  information? 

a)  Systems  design  team 

b)  Production,  Accounting  etc. 

c)  N/A 

d)  Vendors 

e)  R  &  D  function  of  software  development 

f)  Operation 

g)  Most  data  has  to  queried.  The  IT  person  must  understand  and  explain  the  data 

h)  Internal  finance  control 

i)  Business  Analyst 


Conclusion:  Various  answers,  depends  on  the  company  and  department  taking  the  survey. 

Q5.  Could  you  have  waited  longer  to  get  this  information  before  making  changes  in 
your  strategies?  If  yes,  how  long? 

a)  Yes,  a  few  hours 

b)  No 

c)  N/A 

d)  No  need.  Might  as  well  put  it  together  and  see  how  it  operates  on  our  own. 

e)  Yes,  a  few  days 

f)  Yes,  a  week  or  more 

g)  Depends  on  the  severity  of  the  problem.  Time  of  waiting  can  range  from  hours  to  a 
week. 

h)  Yes;  because  we  are  unable  to  guess 

i)  Yes,  a  few  days 


Conclusion:  Most  of  the  companies  can  wait,  but  not  for  very  long.  If  they  have  to  wait  for 
longer,  they  go  ahead  and  change  their  strategies. 


Q6.  What  do  you  estimate  is  the  damage  from  non-assured  information  on  your 
departments’  performance? 

a)  Relatively  insignificant 

b)  Very  significant 

c)  Somewhat  significant 

d)  Relatively  insignificant 

e)  Somewhat  significant 

f)  Very  significant 

g)  Ranges  from  relatively  insignificant  to  very  significant 

h)  Very  significant 

i)  Very  significant 

Conclusion:  4  out  of  the  6  companies  say  it  somewhat  significant  or  very  significant. 

However  2  companies  say  that  it  is  relatively  insignificant. 

Q7.  How  often  do  you  examine  the  relative  significance  (value)  of  the  information  you 
gather/  store/maintain? 

a)  Frequently 

b)  Frequently,  and  it  is  a  lot  of  work 

c)  Occasionally 

d)  Only  at  system  change 

e)  Only  at  system  change 

f)  Not  for  a  long  time 

g)  Occasionally 

h)  Frequently 

i)  Frequently 


Conclusion:  It  depends,  some  companies  do  it  frequently  while  there  are  extreme  cases  too 
when  the  company  does  not  examine  the  relative  significance  of  information  for  a  long  time. 

Q8.  Why  were  you  not  completely  informed  at  the  time  of  making  decisions  to  change 
your  strategies? 

a)  This  company  feels  that 

Sometimes  the  information  had  to  disregarded  due  to 

•  Unreliable  sources 

•  Incomplete  information 

•  Inaccurate  information 

b)  This  company  feels  that 

They  always  had  the  information  available. 

It  is  sometimes  delayed  since  people  need  time  to  send  the  information;  this  is  not  the  main 
problem  though. 

Sometimes  the  information  had  to  be  disregarded  due  to: 

•  Unreliable  sources 


•  Multiple  sources;  the  units  are  messed  up,  hence  making  the  information  inappropriate 

•  Inaccurate  information,  stocks  for  example 

c)  NOT  ANSWERED 

d)  The  information  was  delayed. 

e)  This  company  feels  that: 

Sometimes  information  was  unavailable,  as  it  was  not  measured. 

Sometimes  the  information  had  to  be  disregarded  due  to: 

•  Unreliable  source 

•  Inaccurate  information 

f)  This  company  feels  that: 

Sometimes  the  information  was  not  available  since  nobody  measures  it. 

It  was  delayed  due  to  communication  problems. 

And  sometimes  the  information  was  disregarded  since  it  was  not  accurate  enough. 

g)  Communication  and  unavalilibity  is  always  a  problem  but  the  more  important  problem 
is  when  the  source  is  not  reliable,  unreasonable,  incomplete  data  is  given,  the  format  is 
inappropriate  and  the  information  is  inaccurate. 

h)  Unavailable  information:  IT  systems  not  well  organized 

Data  unavailable  or  missing  or  false  or  misunderstood  by  people  in  charge  of  dealing 
with  them. 

Delayed  information  or  imprecise  information:  useful  information  was  not  available 
for  the  appropriate  decision,  committee  had  to  wait. 

i)  All  three  factors  contributed,  but  most  of  the  problems  were  with  information 
significance. 

Conclusion:  Most  companies  disregard  the  information  due  to  the  fact  that  they  maybe  from 
unreliable  sources  or  the  information  may  be  inaccurate.  Also  communication  problems  seem 
to  be  a  major  issue  for  most  of  them. 


Q9.  Which  type  of  data  is  most  often  affected  by  information  failures  like  delayed  or 
wrong  information? 

a)  Purchase  Order  Processing 

b)  Expenses  by  projects,  levels  of  stocks 

c)  Entry  of  billable  hours  and  expenses 

d)  Research  Projects 

e)  Target  timescales  for  delivery,  resources  required  for  delivery 

f)  Stocks  buffer,  work  orders  dispatch  sequence 

g)  Level  of  stocks,  and  WIP 

h)  Margin;  rentability  by  customers;  sales 

i)  Work  in  progress  and  metrics 

Conclusion:  Various  answers,  depends  on  the  company  and  department  taking  the  survey. 


Q10.  For  which  type  of  data  are  the  consequences  of  information  failure  the  most 
dangerous? 


a)  Selling  prices 

b)  Expenses  by  Projects 

c)  Idle  labor  and  resources 

d)  Large  purchases 

e)  Target  timescales  for  delivery 

f)  Due  dates  for  orders,  sell  actions,  budgeting  update 

g)  Lost  Production,  additional  costs,  excessive  inventory 

h)  Data  related  to  products 

i)  Cost  of  inventory 

Conclusion:  Various  answers,  depends  on  the  company  and  department  taking  the  survey. 


Qll.  Which  performance  measure  of  your  company  is  most  affected  when  an 
information  failure  occurs? 

a)  Consumer  relations,  Profits 

b)  Profits  by  projects 

c)  Profits 

d)  Budgets 

e)  Project  Planning,  Tracking 

f)  Cash  flows,  customer  relations,  purchasing  dep. 

g)  Delivery  performance 

h)  Margins 

i)  Profits  and  metrics  which  lead  people  to  make  bad  decisions  because  of  bad  data 
Conclusion:  Profits  and  consumer  relations 


Q12.  Other  comments  about  recurrent  problems  you  encounter  while  using  the  ERP 
and  related  information  systems  of  the  company: 

a)  Problems  are  diminishing,  quickly  resolvable  or  very  trivial 

b)  Dramatic  improvement  in  quality  of  data- goal! 

c)  NONE 

d)  Surveys  are  not  the  best  way  to  assess  our  problems,  we  want  to  see  an  operational 
solution 

e)  NONE 

fj  Problems  with  our  information  system: 

•  Even  though  we  have  the  ability  to  react  to  changing  situations,  the  decisions  are  not 
made  quick  enough 

•  Each  required  process  change  is  delayed  by  a  long  lead-time  for  acquiring  the 
necessary  information,  analyzing  it  and  arguing  about  its  accuracy 

•  The  company  is  not  fully  utilizing  the  power  of  integrative  information 

d)  Difficult  to  tell  where  you  are  at  and  difficult  to  get  data  and  to  change  processes 
because  of  consequential  damages  due  to  tight  integration. 


e)  More  time  is  spent  on  trying  to  get  the  data  than  in  analyzing  it 

f)  Need  to  be  more  focused  on  process  standardization. 

Conclusion:  Decision-making  is  more  of  a  problem  rather  than  the  methods  to  assure  data. 
Also  they  feel  that  the  whole  process  of  trying  to  assure  data  is  too  time  consuming.  The  goal 
is  to  try  and  achieve  improvement  in  data  quality. 


